Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Anomaly detection method for hydrologic sensor data based on SparkR
LIU Zihao, LI Ling, YE Feng
Journal of Computer Applications    2019, 39 (2): 436-440.   DOI: 10.11772/j.issn.1001-9081.2018081782
Abstract387)      PDF (891KB)(221)       Save
To efficiently detect outliers in massive hydrologic sensor data, an anomaly detection method for hydrological time series based on SparkR was proposed. Firstly, a sliding window and Autoregressive Integrated Moving Average (ARIMA) model were used to forecast the cleaned data on SparkR platform. Then, the confidence interval was calculated for the prediction results, and the results outside the interval range were judged as anomaly data. Finally, based on the detection results, K-Means algorithm was used to cluster the original data, the state transition probability was calculated, and the anomaly data were evaluated in quality. Taking the data of hydrologic sensor obtained from the Chu River as experimental data, experiments on the detection time and outlier detection performance were carried out respectively. The results show that the millions of data calculation by two slaves costs more time than that by one slave, but when calculating the tens of milllions of data, the time costed by two slaves is less than that by one slave, and the maximum reduction is 16.21%. The sensitivity of the evaluation is increased from 5.24% to 92.98%. It shows that under big data platform, the proposed algorithm which is based on the characteristics of hydrological data and combines forecast test and cluster test can effectively improve the computational efficiency of hydrologic time series detection for tens of millions data and has a significant improvement in sensitivity.
Reference | Related Articles | Metrics
Improved D-Nets algorithm with matching quality purification
YE Feng, HONG Zheng, LAI Yizong, ZHAO Yuting, XIE Xianzhi
Journal of Computer Applications    2018, 38 (4): 1121-1126.   DOI: 10.11772/j.issn.1001-9081.2017102394
Abstract384)      PDF (1072KB)(353)       Save
To address the underperformance of feature-based image registration under situations with large affine deformation and similar targets, and reduce the time cost, an improved Descriptor-Nets (D-Nets) algorithm based on matching quality purification was proposed. The feature points were detected by Features From Accelerated Segment Test (FAST) algorithm initially, and then they were filtered according to Harris corner response function and meshing. Furthermore, on the basis of calculating the line-descriptor, a hash table and a vote were constructed, thus rough-matching pairs could be obtained. Eventually, mismatches were eliminated by the purification based on matching quality. Experiments were carried out on Mikolajczyk standard image data set of Oxford University. Results show that the proposed improved D-Nets algorithm has an average registration accuracy of 92.2% and an average time cost of 2.48 s under large variation of scale, parallax and light. Compared to Scale-Invariant Feature Transform (SIFT), Affine-SIFT (ASIFT), original D-Nets algorithms, the improved algorithm has a similar registration accuracy with the original algorithm but with up to 80 times speed boost, and it has the best robustness which significantly outperforms SIFT and ASIFT, which is practical for image registration applications.
Reference | Related Articles | Metrics